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(57) A mark management server (103) embeds a 
mark I D and so on in a specified mark image in response 
to a mark request from a WWW server (102), registers 
information related to a Web page corresponding to this 
mark ID in a mark management DB (2021), and sends 
the mark embedded with the information to the server 
(102). The server (102) attaches this mark to a created 
Web page, and registers the Web page in a Web page 
DB (3021 ). A client terminal (101 ) acquires a Web page 
from the server (102), reads information embedded in a 
mark, and issues a request for a search according to the 
mark to the server (103). The server (103) references 
the DB (2021), and sends information associated with 
a Web page corresponding to a specified mark ID to the 
terminal (101). 
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Des ription 

[0001] The present invention relates generally to an 
information search system for searching for multimedia 
data, and more particularly to a system for searching for 
multimedia data based on fixed-pattern data such as a 
mark mounted in the multimedia data. 
[0002] In recent years, WWW (World Wide Web) sys- 
tems have become popular for utilizing networks such 
as the Internet. A WWW system comprises WWW serv- 
ers each for providing a variety of information, and client 
terminals connected to the WWW servers through the 
Internet for receiving information provided therefrom. 
The respective WWW servers open their unique Web 
pages, such that a user can access a Web page by spec- 
ifying URL (Uniform Resource Locater) corresponding 
to the Web page to a browser program running on a di- 
nt terminal. The browser can display multimedia data 
such as text, image data, video data, audio data and so 
on included in the Web page thus acquired on a display 
device of the client terminal. A Web page is described 
using a structural description language called HTML 
(Hyper Text Mark-up Language), so that another Web 
page can be accessed through its URL set in a Web 
page. 

[0003] When a client searches Web pages with a cer- 
tain purpose, the client must access a Web page of a 
WWW server which provides a search service, and re- 
quest for a key word search or specify a Web page which 
may satisfy the purpose of the client with reference to a 
link list provided by the search service. The link list refers 
to a Web page which collectively has a plurality of listed 
link destinations of Web pages previously selected in 
accordance with an intention of an editor. 
[0004] While a Web page is a copyrighted work, a cli- 
ent, once acquiring the Web page, can readily create 
duplicates and deliver the duplicates through the Inter- 
net. For preventing unauthorized copies of digital data 
sent through the Internet in this way, "electronic water- 
mark" techniques have been proposed for embedding 
ID information, a logomarkorthe like of an author within 
a copyrighted work in secrecy. 

[0005] According to the prior art search techniques 
mentioned above, it is not easy to search Web pages 
published by particular authors, Web pages highly eval- 
uated by Web page evaluating organizations, and so on. 
Even if a search service provided on the WWW is uti- 
lized to conduct a key word search, this may result in a 
problematic situation which presents search missing 
that fails to include desired Web pages in a search re- 
sult, and search noise that includes unwanted informa- 
tion. In addition, when existing link lists cannot be uti- 
lized, a client must create his own link list. However, cre- 
ation of a link list involves describing link data to s I ct- 
d W b pages using the HTML, and so on. This work 
implies a problem in that a large number of work st ps 
is required and the created list cannot flexibly support 
changes in Web pages, if any. Furthermore, it is also 



difficult to distinguish normally published Web pages 
from forged or tampered Web pages. 
[0006] Preferably, It is an object of the present inven- 
tion to provide a system for searching only for normal 
5 multimedia data without relying on a key word search or 
creation of a link list, and a program for executing such 
a search. 

[0007] Multimedia data handled by the present inven- 
tion is assumed to be mounted with fixed-pattern data 
such as a logo mark, which is embedded with code in- 
formation that is difficult for human to perceptually rec- 
ognize. A program on a server for managing such fixed- 
pattern data embeds specified code information in spec- 
ified fixed-pattern data in response to a request from the 
outside, and sends the fixed-pattern data to a requester. 
Then, the program registers a database with information 
related to multimedia data mounted with the code infor- 
mation, in correspondence to the code information, 
searches the database based on specified code infor- 
mation in response to a search request from a client, 
extracts information related to multimedia data corre- 
sponding to the specified code information, and sends 
the extracted information to a requester. A program on 
the client side in turn decodes code information embed- 
ded in fixed-pattern data mounted in multimedia data, 
issues to the management server a search request for 
information related to associated multimedia data based 
on the code information, and displays acquired informa- 
tion related to the associated multimedia data on a dis- 
play device. 

BRIEF DESCRIPTION OF THE DRAWINGS 
[0008] 

Fig. 1 is a block diagram illustrating the configura- 
tion of an information search system according to 
an embodiment of the present invention; 
Fig. 2 is a block diagram illustrating the internal con- 
figuration of a mark management server 103 in the 
embodiment; 

Fig, 3 is a block diagram illustrating the internal con- 
figuration of a WWW server 1 02 in the embodiment; 
Fig. 4 is a blockdiagram illustrating the internal con- 
figuration of a client terminal 101 in the embodi- 
ment; 

Fig. 5 is a table showing a data structure for a mark 
management DB 2021 in the embodiment; 
Fig. 6 shows an example of information embedded 
in a mark; 

Fig. 7 is a flow diagram illustrating a procedure for 
a mark dispense processing in the embodiment; 
Fig. 8 is a flow diagram illustrating a procedure for 
an information search which utiliz s the mark in th 
£5 embodim nt; 

Fig. 9 is a tabl showing anoth r xample of a data 
structure for the mark management DB 2021 ; 
Fig. 10 shows other examples of information em- 
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b dd d in th mark; and 

Fig. 11 is a block diagram illustrating the internal 
configuration of the mark managem nt server 103 
according to another embodiment. 

DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0009] An embodiment of the present invention will 
hereinafter be described with reference to the accom- 
panying drawings. 

[0010] Fig. 1 is a block diagram illustrating the config- 
uration of a system according to this embodiment. The 
system has a plurality of WWW servers 1 02 for providing 
information to a network such as the Internet; a plurality 
of client terminals 101 for accessing information provid- 
ed by the WWW servers; and mark management serv- 
ers 103 for managing a variety of marks, wherein they 
are all interconnected through the network. The mark 
herein referred to means still image data, and is as- 
sumed to be image data which allows the userto visually 
understand what the image data means when it is dis- 
played. For example, the mark may be a logo of a credit 
card company, an award mark issued by a Web page 
evaluating organization, a recommendation mark, a 
symbol mark of a self-governing body, or the like. 
[0011] Fig. 2 is a block diagram illustrating the internal 
configuration of the mark management server 103. The 
mark management server 103 comprises a processing 
unit 201 such as PC (personal computer), WS (worksta- 
tion) or the like; an external storage device 202 such as 
HDD (hard disk drive); input devices 203 such as a key- 
board, a mouse and so on; and an output device 204 
such as a display. The processing unit 201 includes an 
interface 2011 for connection to a network; a processor 
201 2 for processing associated with the mark manage- 
ment server 1 03; a memory 201 3 for temporarily storing 
a program; an internal storage device 2014 for storing 
programs and data; and an interface 2015 for connec- 
tion to input and output devices. The external storage 
device 202 stores a mark management DB (database) 
2021 and a mark image DB 2022. The mark manage- 
ment DB 2021 stores information on Web pages to 
which a mark is attached, while the mark image DB 2022 
stores image data associated with respective marks. 
[001 2] The internal storage device 201 4 stores a mark 
creating program 20143, a mark management DB con- 
trol program 201 41 and an operating system 201 42. The 
mark creating program 20143 fetches a specified mark 
image from the mark image DB 2022 in response to a 
mark transmission request from the WWW server 102, 
embeds the mark image with information such as an 
identifier of the mark, stores the specified mark identifier 
and information on a Web pag to which this mark is 
attach d (mount d) in the mark management DB 2021 
through the mark manag m nt DB control program 
20141, and transmits the mark image embedd d with 
the information to the WWW server 102. 



[0013] Th mark manag ment DB control program 
20141 accesses th mark management DB 2021 in ac- 
cordance with a request from the mark creating program 
20143, and accesses the mark management DB 2021 
s in response to a request for associated information re- 
lated to the same mark from a client terminal 101 to read 
requested information which is sent to the requesting 
client terminal 101. The processing performed by the 
mark management server 103 in accordance with a 
io mark transmission request from the WWW server 102 
for registering the mark management DB 2021 with in- 
formation on a Web page to which a specified mark is 
attached, and transmitting an mark image embedded 
with information to the WWW server 102 is hereinafter 
is called the "mark dispense processing - . 

[0014] Fig. 3 is a block diagram illustrating the internal 
configuration of the WWW server 102. The WWW serv- 
er 1 02 comprises a processing unit 301 such as PC, WS 
or the like; an external storage device 302 such as HDD; 
20 input devices 303 such as a keyboard, a mouse and so 
on; and an output device 304 such as a display. The 
processing unit 301 includes an interface 3011 for con- 
nection to a network; a processor 3012 for processing 
associated with the WWW server 102; a memory 3013 
25 for temporarily storing a program; an internal storage 
device 3014 for storing programs and data; and an in- 
terface 301 5 for connection with input and output devic- 
es. The external storage device 302 stores a Web page 
DB 3021 , while the Web page DB 3021 stores a plurality 
30 of Web pages managed by the WWW server 102. The 
internal storage device 3014 stores a Web page creat- 
ing program 30143, a Web page DB control program 
301 41 , a mark acquiring program 301 44 and an operat- 
ing system 30142. The Web page creating program 
35 30143 creates a Web page in accordance with instruc- 
tions from the user through the input device 303, and 
attaches an acquired mark image to the created Web 
page which is then stored in the Web page DB 3021 
through the Web page DB control program 30141 . The 
40 Web page DB control program 301 41 accesses the Web 
page DB 3021 in accordance with a request from the 
Web page creating program 301 43, as well as accesses 
the Web page DB 3021 in response to a request from a 
client terminal 1 01 to read a requested Web page there- 
45 from, and sends the read Web page to the client terminal 
101 . The mark acquiring program 301 44 issues a mark 
transmission request to the mark management server 
103 in response to a request from the Web page creat- 
ing program 30143, and passes an acquired mark im- 
50 age to the Web page creating program 301 43. 

[0015] Fig. 4 is a block diagram illustrating the internal 
configuration of the client terminal 101 . The client termi- 
nal 101 comprises a processing unit 401 such as PC, 
WS or the lik ; input d vices 402 such as a keyboard, 
55 a mous and so on; and an output d vice 403 such as 
a display. Th proc ssing unit 401 includes an interface 
4011 for connection to a network; a proc ssor 4012 for 
processing associated with the client terminal 101; a 
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memory 4013 for temporarily storing a program; an in- 
ternal storage device 401 4 for storing programs and da- 
ta; and an interface 401 5 for connection to input and out- 
put devices. The internal storage device 4014 stores a 
browser program 40141 , a search processing program 
40143, and an operating system 40142. The browser 
program 40141 issues a Web page transmission re- 
quest to the WWW server 102, and displays an acquired 
Web page on the output device 403. The search 
processing program 40143 receives a search request 
from the browser program 40141 to extract embedded 
information from a specified mark image, issues a 
search request for associated information related to the 
mark to the mark management server 103, edits ac- 
quired information on associated Web pages, and dis- 
plays the edited information on the output device 403. 
Search request processing in the search processing 
program 40143 issues a search request to the mark 
management server 103 in response to a search re- 
quest for associated information related to the mark 
from the browser program 401 41 . Mark information read 
processing decodes information embedded in a mark 
image. Display/edit processing in turn edits and displays 
a list of acquired Web pages associated with the mark. 
[0016] Fig. 5 is a table showing a data structure for 
the mark management DB 2021. Each record in the 
mark management DB 2021 is composed of a mark ID 
501 ; a Web page URL 502 and a page name 503. The 
mark I D 501 is an identifier for each mark image stored 
in the mark image DB 2022. The Web page URL 502 is 
the URL of a Web page to which the mark is attached. 
The page name 503 may be a name, entry information 
or the like which shows the contents of the Web page. 
[0017] Fig. 6 shows an example of information em- 
bedded in a mark image. Each mark image is embedded 
with character strings indicative of a mark ID 601 and a 
mark management server address 602 as code infor- 
mation. The mark ID 601 is an identifier of each mark 
image, and the mark management server address 602 
indicates the address of the mark management server 
1 03 that has embedded information in the mark. 
[001 8] Fig. 7 is a flow diagram illustrating a procedure 
for the mark dispense processing. The Web page cre- 
ating program 301 43 of the WWW server 1 02 interacts 
with a manager of a WWW server through the input de- 
vice 303 and the output device 304 to create a Web 
page, and stores the created Web page in the Web page 
DB 3021 through the Web page DB control program 
30141 (step 701). In this event, an access permission 
for the Web page stored in the Web page DB 3021 is 
not granted to ordinary users so that they cannot access 
this Web page. Next, the mark acquiring program 301 44 
issues a request to the mark management server 103 
to send a mark to be attach d to the created Web pag 
(step 702). This sending requ st includes a mark ID, a 
Web page URL and a pag name. The mark creating 
program 201 43 of the mark management server 1 03 au- 
thenticates the user for this sending request, and th n 



acquires a pertinent Web page based on th specified 
URL, and displays the Web page on the output device 
204 for allowing the operator of the mark management 
server 103to confirm the contents of the Web page (step 
5 703). When the operator requests to display a mark im- 
age, the mark creating program 20143 accesses the 
mark image DB 2022 based on a received mark ID, and 
displays the mark image associated with the mark ID on 
the output device 204. After the operator determines 
10 that the mark may be dispensed to the Web page, for 
which the mark has been requested, without problem, 
when a mark dispensing instruction is issued through 
the input device 203, the mark creating program 20143 
embeds the mark ID 601 and the mark management 
is server address 602 in the requested mark image (step 
704). A technique for embedding particular information 
in a mark image is known as "digital watermark" which 
is described, for example, in Nikkei Electronics, pp. 
100-107, No. 683, 1997. Next, the mark creating pro- 
20 gram 20 1 43 registers the mark ID 501 of the dispensed 
mark image, and the URL 502 and the page name 503 
of the requested Web page in the mark management 
DB 2021 (step 705). Next, the mark creating program 
20143 transmits the mark image embedded with the in- 
2S formation to the WWW server 1 02 (step 706). 

[0019] The mark acquiring program 30144 of the 
WWW server 102, which has received the mark image, 
passes it to the Web page creating program 30143. The 
Web page creating program 30143 attaches this mark 
30 image to a created Web page to create a marked Web 
page (step 707). For attaching the mark, the Web page 
and the mark image are displayed on the output device 
304, and the mark is attached at a location specified by 
the manager on the Web page. Next, the pertinent Web 
35 page previously registered on the Web page DB 3021 
is updated with the marked Web page (step 708). Then, 
the access permission for the updated Web page is 
changed to be accessible by ordinary users. 
[0020] Fig. 8 is a flow diagram illustrating a procedure 
40 for information search utilizing a mark. It should be noted 
that the information search herein mentioned refers to 
the processing for searching for Web pages which have 
attached thereon the same mark as a mark attached on 
a certain Web page, dispensed from the mark manage- 
rs ment server 103. The browser program 40141 of the cli- 
ent terminal 101 requests a pertinent WWW server 102 
to transmit a Web page based on a URL specified 
through the input device 402 (step 801). The Web page 
DB control program 30141 of the WWW server 102 re- 
50 ceives this transmission request, searches the Web 
page DB 3021 (step 802), retrieves the specified Web 
page, and transmits the contents of that Web page to 
the client terminal 101 (step 803). Assume that the re- 
trieved W b page has a mark attached thereon, dis- 
ss pensed from the mark managem nt serv r 103. The 
brows r program 40141 of the client terminal 101 dis- 
plays the receiv d Web page on the output devic 403 
(step 804). The user can visually recognize what the 



4 



oMcrw^irv -CD noocm *o I ^ 



t 



7 



EP 0 936 531 A2 



8 



mark display d on the output device 403 repres nts.but 
cannot visually recognize information embedded in the 
mark. When a search request is made for associated 
information by specifying a mark attached on a Web 
page through the input device 402, the search request 
processing is started in the search processing program 
40143 (step 805). This processing in the search 
processing program 401 43 starts the mark information 
read processing. 

[0021] The mark information read processing reads 
information (the mark ID 601 and the mark management 
server address 602) embedded in a mark image (step 
806). The technique for reading text information from a 
mark image is known as a part of the "digital watermark 0 
technique. Next, the search request processing re- 
quests the mark management server 103 to search for 
associated information based on the information em- 
bedded in the mark (step 807). Upon receipt of the re- 
quest, the mark management DB control program 
20141 of the mark management server 103 searches 
the mark management DB 2021 to acquire the Web 
page URLs 502 and the page names 503 of Web pages 
(having the same mark ID 501 as the received mark) to 
which the same mark as the specified one has been dis- 
pensed (step 808). Next, the mark management DB 
control program 20141 transmits a list of the Web page 
URLs 502 and the page names 503, which have been 
acquired as the associated information, to the client ter- 
minal 101 (step 809). The search processing program 
401 43 of the client terminal 1 01 edits the received infor- 
mation with its display/edit processing, and displays the 
edited information on the output device 403 through the 
browser program 40141 (step 810). 
[0022] When any Web page within the associated in- 
formation is requested for display through the input de- 
vice 402 (step 811), the search processing program 
40143 passes the URL of a selected Web page to the 
browser program 40141, and the browser program 
40141 requests the WWW server 102 associated with 
the URL to transmit the Web page (step 81 2). The Web 
DB control program 30141 of the WWW server 102 
searches the Web page DB 3021 to retrieve the speci- 
fied Web page, and sends the contents thereof to the 
client terminal 101 (step 813). The browser program 
40141 of the client terminal 101 displays the received 
Web page on the output device 403 (step 814). 
[0023] As described above, since the mark manage- 
ment sever 1 03 holds mark dispense information on the 
mark management DB 2021 implemented therein, the 
mark management server 1 03 can specify a destination 
to which the same mark has been dispensed, and there- 
fore allows the mark ID 501 to be used as a key for 
searching for the associated information. 
[0024] While th user could duplicat a mark image 
within a Web page acquired from the WWW s rver 102 
and attach the duplicate to a Web page cr at dther by 
without authorization, such a Web page is not registered 
in the mark management server 1 03 and therefore is not 



regard d as an object of a s arch for th associat d in- 
f rmation as described above. In addition, for a Web 
page on which a mark thus forged is attached, an ille- 
gally duplicated mark can be detected by referring to the 

s mark management server 1 03 based on the mark I D and 
the Web page URL thereof. Furthermore, since the in- 
formation embedded in a mark image is difficult to 
tamper, it is difficult for an unauthorized user to modify 
information embedded in a normal mark image to inf or- 

10 matbn convenient to him. It is therefore possible to pre- 
vent the action of distributing false marks forged from 
an authoritative mark, which has been famous, over a 
network. 

[0025] As described above, since the client terminal 
75 101 decodes information embedded in a mark, this in- 
formation is likely to be acquired and abused by the user. 
As a method of ensuring the security for embedded in- 
formation, a method of encrypting the embedded infor- 
mation is contemplated. For example, the mark man- 
20 agement server 1 03 may encrypt information to be em- 
bedded in a mark using a secret key, prior to the em- 
bedding of the information in the mark, in the processing 
for embedding information in a mark (step 704). The cli- 
ent terminal 101 requests the mark management server 
25 103 to send a public key, and the mark management 
server 103 responsively sends the public key to the cli- 
ent terminal 101. The client terminal 101 acquires the 
encrypted code information embedded in the mark from 
the mark management server 103, and decrypts the 
30 code information embedded in the mark using the public 
key before utilizing the code information. While the us- 
ers can relatively easily acquire this public key, the users 
can be managed through user authentication. 
[0026] When the same mark has been dispensed to 
35 a large number of Web servers, attribute information 
may be added to each record in the mark management 
DB 2021 such that the attribute information is utilized for 
searching for a mark. For example, an entry called "in- 
tended age" may be provided as attribute information, 
40 wherein a search result may be narrowed up by speci- 
fying the attribute as a search condition if Web pages 
are classified into those "for 20 years or more" and those 
lor under 20 years". Also, as illustrated in Fig. 9, an en- 
try called 'order" 504 may be added as attribute infor- 
ms mation in each record of the mark management DB 
2021 in addition to the mark ID 501 , the Web page URL 
502 and the page name 503. In this case, the mark man- 
agement DB control program 201411 may refer to the 
order 504 in the mark management DB 2021 to sort per- 
50 tinent records according to the order 504 used as a key, 
and then send a search result to the client terminal 1 01 , 
thereby making it possible to display the search result 
in the order set in the mark management DB 2021. 
[0027] Of cours , th mark management DB control 
55 program 20141 may transfer pertinent records in the 
mark management DB 2021 as they ar toth client ter- 
minal 101, so that the client terminal 101 can sort the 
records according to the order 504 used as a key. With 
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such a method, Web pages having the same mark ar 
browsed by the user of the client terminal 1 01 in an order 
intended by a mark dispenser. 

[0028] Altemativ ty, a mark may b embedded with 
the position in the order at which an associated Web 
page is browsed, instead of providing the order 504 in 
each record in the mark management DB 2021 . Fig. 1 0 
shows exemplary records which have information la- 
belied front 603 and behind 604 embedded therein in 
addition to the mark ID 601 and the mark management 
server address 602, as information to be embedded in 
a mark. Each line in Fig. 1 0 corresponds to each of Web 
pages having the same mark, wherein front 603 sets the 
URL and page name of a Web page positioned imme- 
diately in front of a current Web page, while behind 604 
sets the URL and page name of a Web page positioned 
immediately behind the current Web page. With this 
method, the mark management server 103 is only re- 
quired to send the client terminal 101 the Web page URL 
502 and the page name 503 of the record at the head 
of a group of Web pages having the same mark, and the 
client terminal 101 can directly access a Web page po- 
sitioned in front of or behind a displayed Web page. 
[0029] While the foregoing embodiment has shown 
an example of searching for associated information for 
a single mark, the client terminal 101 may simultane- 
ously issue search requests to a plurality of marks, so 
that a plurality of marks may be dispensed respectively 
from different mark management servers 103 without in- 
curring any problem. 

[0030] Also, while in the foregoing embodiment, the 
mark management server 1 03 notifies the client terminal 
101 of destinations to which a specified mark has been 
dispensed, the mark management server 103 may also 
transfer the contents of the mark management DB 2021 
to the client terminal 1 01 utilizing techniques such as file 
transfer or the like, in response to desire to know which 
marks can be otherwise seen, or to make a search for 
associated informatbn for another mark. Fig. 11 illus- 
trates the internal configuration of a client terminal 101 
which handles a transferred mark management DB 
2021 : An internal storage device 4014 is provided there- 
in with a mark information storage area 40145 for storing 
the contents of the mark management DB 2021 , and a 
search processing program 40143 is provided therein 
with mark information search processing as a program 
for searching the mark information storage area 40145. 
[0031] While in the foregoing embodiment, the mark 
management DB 2021 is located on the mark manage- 
ment server 103, it may be located on any other data- 
base server which may be accessible by the mark man- 
agement server 103, instead of locating it on the mark 
management server 103. Also, while in the foregoing 
embodim nt, th mark manag m nt s rver 103 p r- 
forms the mark disp ns processing and th s arch 
proc ssing, th mark manag m nt s rver 103 may be 
dedicated only to the mark dispense processing, while 
the search processing may be assigned to another serv- 
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er having duplicat d contents of the mark management 
DB 2021 , in view of the loading on the servers. In this 
case, the addr ss of a search server may be mbedded 
in a mark instead of embedding the address of the mark 
5 management server 103. Also, in this case, a plurality 
of search servers may be provided corresponding to the 
single mark management server 103 for distributing the 
load of the search processing. 

[0032] While the foregoing embodiment has been de- 

10 scribed in connection with marks implemented by still 
image data, the present invention is not limited to this 
particular type of marks. When perceptually recogniza- 
ble data formed of a single or a combination of a plurality 
of character data (text information), still image data, vid- 

15 eo data (moving image data) and audio data is called 
multimedia data, the present invention can be applied 
to fixed-pattern data that may constitute a portion of 
such multimedia data as a mark. For example, the 
present invention can be applied to the case where the 

20 multimedia data is media data such as a movie including 
video data and audio data, and a mark is a fixed-pattern 
data that constitutes a portion of the video data or the 
audio data. As an example of embedding secret infor- 
mation in character data, there is such a method that 

25 adds a predetermined number of blanks between char- 
acters which constitutes text information, or behind the 
character string. Also, a technique is known for embed- 
ding secret information in audio information so as not to 
be audibly recognizable. When video data or audio data 

30 is applied as a mark, this may exceed the concept of a 
normal mark, or the concept of attaching a mark to a 
Web page. In this case, however, what is equivalent to 
a Web page may be corresponded to the body of multi- 
media data including entry information such as a title, a 

35 publisher or the like; the mark to fixed-pattern data 
mounted in the multimedia data body; and the mark ID 
to an identifier of this fixed-pattern data (ar; identifier of 
a copyright owner or the like). 

[0033] According to the present invention as de- 
40 scribed above, since information on associated multi- 
media data stored in a database is extracted on the ba- 
sis of code information embedded in fixed-pattern data 
mounted in the multimedia data, targeted normal multi- 
media data can be searched for without causing search 
45 missing or search noise associated with a key word 
search or without requiring a large number of steps 
which would be required to create a link list. 



50 Claims 

1- An information search processing method for reg- 
istering information related to multimedia data hav- 
ing code information associat d th r with in a da- 
55 tabas (2021), said method comprising th st psof: 

(a) in respons to a r quest from the outside, 
embedding perceptually unrecognizable code 
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information in p rceptually recognizable fixed- 
pattern data mounted in multimedia data in- 
cluding at least one of character data, still im- 
age data, moving image data and audio data, 
and sending the multimedia data embedded s 
with the code information to a requester (102); 

(b) registering information related to said mul- 
timedia data mounted with said code informa- 
tion in the database, in correspondence to said 
code information; and 10 

(c) in response to a search request from the out- 
side, searching said database based on spec- 
ified code information to extract information on 
said multimedia data corresponding to the 
specified code information, and sending the ex- *s 
tracted information to a requester. 

2. An information search processing method accord- 
ing to claim 1 , wherein said fixed-pattern data, em- 
bedded with said code information, is a mark image 20 
in the form of a still image. 

3. An information search method for requesting to 
search for associated multimedia data based on 
perceptually unrecognizable code information em- 2s 
bedded in perceptually recognizable fixed-pattern 
data mounted in multimedia data including at least 
one of character data, still image data, moving im- 
age data and audio data, said method comprising 

the steps of: 30 

(a) decoding said code information embedded 
in said fixed-pattern data related to the multi- 
media data; 

(b) issuing a search request for information re- 3S 
lated to the associated multimedia data to an 
external processing unit (103) based on said 
code information, said external processing unit 
having information related to the associated 
multimedia data corresponding to said code in- 40 
formation as a database (2021); and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 

45 

4. An information search processing method for re- 
questing to search for associated multimedia data 
based on visually unrecognizable code information 
embedded in a visually recognizable mark image 
mounted in multimedia data including at least one so 
of character data, still image data, moving image 
data and audio data, said method comprising the 
steps of: 

(a) displaying said mark imag on a display d - ss 
vice (403); 

(b) issuing to an external processing unit (103) 
a search request for information relat d to the 



associat d multimedia data based on said code 
information embedded when said mark image 
was specified; and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 

5. An information search system having a first server 
(102) for providing multimedia data including at 
least one of character data, still image data, moving 
image data and audio data, a client terminal (101) 
for receiving the multimedia data provided by said 
first server, and a second server (103) for embed- 
ding perceptually unrecognizable code information 
in perceptually recognizable fixed-pattern data 
mounted in the multimedia data and for managing 
information related to the multimedia data mounted 
with said code information, wherein: 

said first server includes means (30141) re- 
sponsive to a request from said client terminal 
for sending specified multimedia data to said 
client terminal, means (30144) for requesting 
said second server to send said fixed-pattern 
data embedded with said code information, and 
means (30143) for mounting said fixed-pattern 
data in corresponding multimedia data; 
said second server includes means (20143) re- 
sponsive to a request from said first server for 
mounting specified code information in speci- 
fied fixed-pattern data and sending the fixed- 
pattern data to said first server, means (201 43) 
for registering information related to multimedia 
data mounted with said code information in a 
database, and means (20141) responsive to a 
search request from said client terminal for 
searching said database based on specified 
code information to extract information related 
to multimedia data corresponding to the speci- 
fied code information and sending the extracted 
information to said client terminal; and 
said client terminal includes means (401 41 ) for 
requesting said first server to acquire the mul- 
timedia data mounted with the fixed-pattern da- 
ta, means (40143) for decoding code informa- 
tion embedded in acquired fixed-pattern data, 
means (401 43) for issuing to said second serv- 
er a search request for information related to 
multimedia data associated with the code infor- 
mation based on the decoded code informa- 
tion, and means (40143) for displaying ac- 
quired information related to the associated 
multimedia data on a display device (403). 

6. A comput r program embodied on a compute r- 
readabl medium to p rform method st ps for r g- 
istering in a database (2021) information related to 
multimedia data having code information associat- 
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d th rewith, said method steps comprising the 
steps of: 

(a) in response to a request from the outside, 
embedding perceptually unrecognizable code s 
information in perceptually recognizable fixed- 
pattern data mounted in multimedia data in- 
cluding at least one of character data, stitl im- 
age data, moving image data and audio data, 
and sending the multimedia data embedded 10 
with the code information to a requester (102); 

(b) registering information related to said mul- 
timedia data mounted with said code informa- 
tion in the database, in correspondence to said 
code information; and is 

(c) in response to a search request from the out- 
side, searching said database based on spec- 
ified code information to extract information on 
said multimedia data corresponding to the 
specified code information, and sending the ex- 20 
tract ed information to a requester. 

7. A computer program embodied on a computer- 
readable medium to perform method steps for re- 
questing to search for associated multimedia data 2s 
based on perceptually unrecognizable code infor- 
mation embedded in perceptually recognizable 
fixed-pattern data mounted in multimedia data in- 
cluding at least one of character data, still image 
data, movin g image data and audio data, said meth- 30 
od steps comprising the steps of: 

(a) decoding said code information embedded 
in said fixed-pattern data related to the multi- 
media data; 35 

(b) issuing a search request for information re- 
lated to the associated multimedia data to an 
external processing unit (103) based on said 
code information, said external processing unit 
having information related to the associated 40 
multimedia data corresponding to said code in- 
formation as a database (2021 ); and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 45 

8. A computer program embodied on a computer- 
readable medium to perform method steps for re- 
questing to search for associated multimedia data 
based on visually unrecognizable code information 50 
embedded in a visually recognizable mark image 
mounted in multimedia data including at least one 

of character data, still image data, moving image 
data and audio data, said method st ps comprising 
th steps of: 55 

(a) displaying said mark image on a display de- 
vice (403); 



(b) issuing to an external processing unit (103) 
a search request for information related to the 
associated multimedia data based on said code 
information mbedded when said mark image 
was specified; and 

(c) displaying acquired information related to 
the associated multimedia data on a display de- 
vice (403). 
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